If we compare the absolute deviation across method pairs and whether or not SAI holds for a model, the following plot shows that for models for which SAI does not hold (i.e., the pair clustering model) the absolute deviation is on average larger. The red points is the mean in each plot. Consequently, all following analyses are based on the data excluding the non SAI (i.e., pair clustering) model.
| model2 | dataset | parameter | cond_x | cond_y | abs_dev | se_c | rhos_max | x | y | prop_ns_noasy | prop_ns_trait | prop_ns_nopb | prop_ns_nonpb | prop_ns | z_abs_dev |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-BAYES | CP-MLE | 0.189 | 0.021 | 0.034 | 0.200 | 0.011 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 8.915 |
| pm | Smith & Hunt 2013.csv:pm | pm:M | NP-BAYES | CP-MLE | 0.159 | 0.019 | 0.081 | 0.831 | 0.991 | 0.000 | 0.000 | 0.200 | 0.160 | 0.090 | 8.501 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.091 | 0.013 | 0.025 | 0.390 | 0.300 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 7.065 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-BAYES | CP-MLE | 0.053 | 0.009 | 0.047 | 0.070 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 5.906 |
| quad | CalanchiniEtAl2014_PI_ableMCMC:quad | quad:D1 | PP-LT-NC | CP-MLE | 0.035 | 0.007 | 0.091 | 0.887 | 0.852 | 0.230 | 0.060 | 0.190 | 0.025 | 0.126 | 5.028 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | PP-B | CP-MLE | 0.106 | 0.021 | 0.025 | 0.406 | 0.300 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 4.994 |
| pm | Smithetal2014_Exp2.csv:pm | pm:C2 | NP-BAYES | CP-MLE | 0.059 | 0.013 | 0.071 | 0.837 | 0.895 | 0.000 | 0.000 | 0.067 | 0.022 | 0.022 | 4.400 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.039 | 0.010 | 0.007 | 0.789 | 0.828 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.815 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-LT-NC | CP-MLE | 0.054 | 0.016 | 0.007 | 0.882 | 0.828 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.455 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-LT-C | CP-MLE | 0.053 | 0.016 | 0.007 | 0.881 | 0.828 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.398 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-MLE | CP-MLE | 0.078 | 0.024 | 0.025 | 0.377 | 0.300 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 3.177 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-PB | CP-MLE | 0.078 | 0.024 | 0.025 | 0.377 | 0.300 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 3.177 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-NPB | CP-MLE | 0.078 | 0.024 | 0.025 | 0.377 | 0.300 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 3.177 |
| hb | Bayen2006Exp1.csv:hb | hb:c | PP-B | CP-MLE | 0.017 | 0.005 | 0.034 | 0.028 | 0.011 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 3.095 |
| rm | HilbigPohl09exp3.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.044 | 0.015 | 0.086 | 0.716 | 0.760 | 0.048 | 0.016 | 0.032 | 0.000 | 0.024 | 2.956 |
| rm | Horn Ruggeri Pachur 2016 .csv:rm | rm:r | NP-BAYES | CP-MLE | 0.066 | 0.024 | 0.086 | 0.327 | 0.262 | 0.167 | 0.056 | 0.111 | 0.000 | 0.083 | 2.754 |
| quad | CalanchiniEtAl2014_PI_ableMCMC:quad | quad:D1 | NP-MLE | CP-MLE | 0.020 | 0.007 | 0.091 | 0.872 | 0.852 | 0.230 | 0.060 | 0.190 | 0.025 | 0.126 | 2.754 |
| quad | CalanchiniEtAl2014_PI_ableMCMC:quad | quad:D1 | NP-PB | CP-MLE | 0.020 | 0.007 | 0.091 | 0.872 | 0.852 | 0.230 | 0.060 | 0.190 | 0.025 | 0.126 | 2.731 |
| quad | CalanchiniEtAl2014_PI_ableMCMC:quad | quad:D1 | NP-NPB | CP-MLE | 0.020 | 0.007 | 0.091 | 0.872 | 0.852 | 0.230 | 0.060 | 0.190 | 0.025 | 0.126 | 2.731 |
| quad | JinEtAl2016s2:quad | quad:ACbb1 | NP-BAYES | CP-MLE | 0.046 | 0.017 | 0.095 | 0.133 | 0.086 | 0.130 | 0.000 | 0.000 | 0.000 | 0.033 | 2.697 |
| rm | Michalkiewicz Erdfelder 2016_Exp.2_week group_test1.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.030 | 0.011 | 0.035 | 0.574 | 0.604 | 0.071 | 0.048 | 0.071 | 0.000 | 0.048 | 2.636 |
| quad | GonsalkoraleEtAl2011:quad | quad:ACbb1 | NP-BAYES | CP-MLE | 0.046 | 0.019 | 0.095 | 0.136 | 0.089 | 0.174 | 0.043 | 0.000 | 0.000 | 0.054 | 2.467 |
| pm | Smith & Hunt 2013.csv:pm | pm:M | PP-B | CP-MLE | 0.030 | 0.012 | 0.081 | 0.961 | 0.991 | 0.000 | 0.000 | 0.200 | 0.160 | 0.090 | 2.416 |
| pm | Smithetal2014_Exp2.csv:pm | pm:C1 | NP-BAYES | CP-MLE | 0.045 | 0.019 | 0.095 | 0.664 | 0.710 | 0.000 | 0.000 | 0.067 | 0.044 | 0.028 | 2.342 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-B | CP-MLE | 0.024 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.300 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-MLE | CP-MLE | 0.025 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.160 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-PB | CP-MLE | 0.025 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.160 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-NPB | CP-MLE | 0.025 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.160 |
| rm | Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.016 | 0.009 | 0.019 | 0.606 | 0.622 | 0.172 | 0.086 | 0.155 | 0.000 | 0.103 | 1.894 |
| rm | HilbigPohl09exp2.csv:rm | rm:a | NP-BAYES | CP-MLE | 0.011 | 0.006 | 0.066 | 0.785 | 0.796 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 1.885 |
| model2 | dataset | parameter | cond_x | cond_y | abs_dev | se_c | rhos_max | x | y | prop_ns_noasy | prop_ns_trait | prop_ns_nopb | prop_ns_nonpb | prop_ns | z_abs_dev |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.091 | 0.013 | 0.025 | 0.390 | 0.300 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 7.065 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-BAYES | CP-MLE | 0.053 | 0.009 | 0.047 | 0.070 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 5.906 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.039 | 0.010 | 0.007 | 0.789 | 0.828 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.815 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-LT-NC | CP-MLE | 0.054 | 0.016 | 0.007 | 0.882 | 0.828 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.455 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-LT-C | CP-MLE | 0.053 | 0.016 | 0.007 | 0.881 | 0.828 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.398 |
| hb | Bayen2006Exp1.csv:hb | hb:c | PP-B | CP-MLE | 0.017 | 0.005 | 0.034 | 0.028 | 0.011 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 3.095 |
| rm | Michalkiewicz Erdfelder 2016_Exp.2_week group_test1.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.030 | 0.011 | 0.035 | 0.574 | 0.604 | 0.071 | 0.048 | 0.071 | 0.000 | 0.048 | 2.636 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-B | CP-MLE | 0.024 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.300 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-MLE | CP-MLE | 0.025 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.160 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-PB | CP-MLE | 0.025 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.160 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-NPB | CP-MLE | 0.025 | 0.011 | 0.047 | 0.042 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 2.160 |
| rm | Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.016 | 0.009 | 0.019 | 0.606 | 0.622 | 0.172 | 0.086 | 0.155 | 0.000 | 0.103 | 1.894 |
| quad | JinEtAl2016s2:quad | quad:ACbb1 | NP-BAYES | CP-MLE | 0.031 | 0.017 | 0.044 | 0.156 | 0.126 | 0.034 | 0.000 | 0.000 | 0.000 | 0.009 | 1.809 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:A | NP-BAYES | CP-MLE | 0.010 | 0.006 | 0.047 | 0.244 | 0.254 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 1.695 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.031 | 0.019 | 0.035 | 0.031 | 0.000 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 1.668 |
| rm | HilbigPohl09exp2.csv:rm | rm:b | NP-BAYES | CP-MLE | 0.012 | 0.007 | 0.029 | 0.671 | 0.684 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 1.662 |
| rm | Michalkiewicz Erdfelder 2016_Exp.2_week group_test2.csv:rm | rm:r | NP-BAYES | CP-MLE | 0.015 | 0.009 | 0.028 | 0.713 | 0.728 | 0.190 | 0.071 | 0.190 | 0.000 | 0.113 | 1.634 |
| pm | SmithBayen2005_Experiment2.csv:pm | pm:C1 | NP-BAYES | CP-MLE | 0.025 | 0.020 | 0.030 | 0.834 | 0.860 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 1.302 |
| 2htsm_4 | S2002_E2:2htsm_4 | 2htsm_4:D | NP-BAYES | CP-MLE | 0.015 | 0.013 | 0.042 | 0.904 | 0.919 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 1.137 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-LT-NC | CP-MLE | 0.010 | 0.009 | 0.047 | 0.008 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 1.111 |
| rm | Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm | rm:a | NP-BAYES | CP-MLE | 0.006 | 0.006 | 0.043 | 0.528 | 0.522 | 0.172 | 0.086 | 0.155 | 0.000 | 0.103 | 1.010 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-MLE | CP-MLE | 0.004 | 0.004 | 0.034 | 0.008 | 0.011 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 1.003 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-PB | CP-MLE | 0.004 | 0.004 | 0.034 | 0.008 | 0.011 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 1.003 |
| hb | Bayen2006Exp1.csv:hb | hb:c | NP-NPB | CP-MLE | 0.004 | 0.004 | 0.034 | 0.008 | 0.011 | 0.038 | 0.000 | 0.038 | 0.000 | 0.019 | 1.003 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | PP-B | CP-MLE | 0.016 | 0.017 | 0.035 | 0.016 | 0.000 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 0.933 |
| rm | Hilbig et al 2015_exp3.csv:rm | rm:a | NP-BAYES | CP-MLE | 0.013 | 0.014 | 0.017 | 0.690 | 0.703 | 0.062 | 0.000 | 0.062 | 0.000 | 0.031 | 0.932 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-LT-C | CP-MLE | 0.008 | 0.009 | 0.047 | 0.009 | 0.018 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 0.928 |
| rm | HilbigPohl09exp2.csv:rm | rm:b | NP-MLE | CP-MLE | 0.008 | 0.009 | 0.029 | 0.675 | 0.684 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 0.905 |
| rm | HilbigPohl09exp2.csv:rm | rm:b | NP-PB | CP-MLE | 0.008 | 0.009 | 0.029 | 0.675 | 0.684 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 0.905 |
| rm | HilbigPohl09exp2.csv:rm | rm:b | NP-NPB | CP-MLE | 0.008 | 0.009 | 0.029 | 0.675 | 0.684 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 0.905 |
| model2 | dataset | parameter | cond_x | cond_y | abs_dev | se_c | rhos_max | x | y | prop_ns_noasy | prop_ns_trait | prop_ns_nopb | prop_ns_nonpb | prop_ns | z_abs_dev |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| hb | Coolin2015.csv:hb | hb:c | NP-BAYES | PP-LT-C | 0.246 | 0.016 | 0.061 | 0.248 | 0.002 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 15.572 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-BAYES | PP-LT-C | 0.061 | 0.007 | 0.047 | 0.070 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 8.282 |
| pm | Smith & Hunt 2013.csv:pm | pm:M | NP-BAYES | PP-LT-C | 0.154 | 0.020 | 0.081 | 0.831 | 0.985 | 0.000 | 0.000 | 0.200 | 0.160 | 0.090 | 7.636 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.030 | 0.005 | 0.035 | 0.031 | 0.001 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 6.217 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.092 | 0.016 | 0.007 | 0.789 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 5.929 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | PP-B | PP-LT-C | 0.015 | 0.003 | 0.035 | 0.016 | 0.001 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 4.708 |
| pm | Smithetal2014_Exp2.csv:pm | pm:C2 | NP-BAYES | PP-LT-C | 0.066 | 0.015 | 0.071 | 0.837 | 0.902 | 0.000 | 0.000 | 0.067 | 0.022 | 0.022 | 4.365 |
| hb | Coolin2015.csv:hb | hb:c | PP-B | PP-LT-C | 0.016 | 0.004 | 0.061 | 0.018 | 0.002 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 4.097 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-B | PP-LT-C | 0.033 | 0.009 | 0.047 | 0.042 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.624 |
| rm | HilbigPohl09exp3.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.076 | 0.021 | 0.086 | 0.716 | 0.792 | 0.048 | 0.016 | 0.032 | 0.000 | 0.024 | 3.559 |
| rm | Pohl et al 2017.csv:rm | rm:r | CP-Bayes | PP-LT-C | 0.054 | 0.016 | 0.007 | 0.827 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.425 |
| rm | Pohl et al 2017.csv:rm | rm:r | CP-MLE | PP-LT-C | 0.053 | 0.016 | 0.007 | 0.828 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.398 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-MLE | PP-LT-C | 0.033 | 0.010 | 0.047 | 0.042 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.350 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-PB | PP-LT-C | 0.033 | 0.010 | 0.047 | 0.042 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.350 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-NPB | PP-LT-C | 0.033 | 0.010 | 0.047 | 0.042 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.350 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-PB | PP-LT-C | 0.061 | 0.021 | 0.007 | 0.820 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.873 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-NPB | PP-LT-C | 0.061 | 0.021 | 0.007 | 0.820 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.873 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-MLE | PP-LT-C | 0.062 | 0.021 | 0.007 | 0.820 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.873 |
| quad | JinEtAl2016s2:quad | quad:ACbb1 | NP-BAYES | PP-LT-C | 0.051 | 0.018 | 0.095 | 0.133 | 0.082 | 0.130 | 0.000 | 0.000 | 0.000 | 0.033 | 2.873 |
| quad | GonsalkoraleEtAl2011:quad | quad:ACbb1 | NP-BAYES | PP-LT-C | 0.062 | 0.022 | 0.095 | 0.136 | 0.074 | 0.174 | 0.043 | 0.000 | 0.000 | 0.054 | 2.872 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-B | PP-LT-C | 0.056 | 0.020 | 0.007 | 0.825 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.836 |
| pm | Smithetal2014_Exp2.csv:pm | pm:C1 | NP-BAYES | PP-LT-C | 0.058 | 0.024 | 0.095 | 0.664 | 0.722 | 0.000 | 0.000 | 0.067 | 0.044 | 0.028 | 2.384 |
| hb | Coolin2015.csv:hb | hb:c | NP-PB | PP-LT-C | 0.014 | 0.007 | 0.061 | 0.016 | 0.002 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 2.126 |
| hb | Coolin2015.csv:hb | hb:c | NP-NPB | PP-LT-C | 0.014 | 0.007 | 0.061 | 0.016 | 0.002 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 2.126 |
| quad | JinEtAl2016s2:quad | quad:ACbb1 | NP-BAYES | PP-LT-C | 0.035 | 0.018 | 0.044 | 0.156 | 0.121 | 0.034 | 0.000 | 0.000 | 0.000 | 0.009 | 1.988 |
| rm | HilbigPohl09exp2.csv:rm | rm:a | NP-BAYES | PP-LT-C | 0.014 | 0.007 | 0.066 | 0.785 | 0.799 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 1.916 |
| hb | Coolin2015.csv:hb | hb:c | NP-MLE | PP-LT-C | 0.002 | 0.001 | 0.061 | 0.000 | 0.002 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 1.794 |
| rm | Michalkiewicz Erdfelder 2016_Exp.2_week group_test2.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.042 | 0.023 | 0.028 | 0.713 | 0.755 | 0.190 | 0.071 | 0.190 | 0.000 | 0.113 | 1.782 |
| pm | Smith & Hunt 2013.csv:pm | pm:M | PP-B | PP-LT-C | 0.024 | 0.014 | 0.081 | 0.961 | 0.985 | 0.000 | 0.000 | 0.200 | 0.160 | 0.090 | 1.770 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-MLE | PP-LT-C | 0.001 | 0.001 | 0.035 | 0.000 | 0.001 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 1.755 |
| model2 | dataset | parameter | cond_x | cond_y | abs_dev | se_c | rhos_max | x | y | prop_ns_noasy | prop_ns_trait | prop_ns_nopb | prop_ns_nonpb | prop_ns | z_abs_dev |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-BAYES | PP-LT-C | 0.061 | 0.007 | 0.047 | 0.070 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 8.282 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.030 | 0.005 | 0.035 | 0.031 | 0.001 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 6.217 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.092 | 0.016 | 0.007 | 0.789 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 5.929 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | PP-B | PP-LT-C | 0.015 | 0.003 | 0.035 | 0.016 | 0.001 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 4.708 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-B | PP-LT-C | 0.033 | 0.009 | 0.047 | 0.042 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.624 |
| rm | Pohl et al 2017.csv:rm | rm:r | CP-Bayes | PP-LT-C | 0.054 | 0.016 | 0.007 | 0.827 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.425 |
| rm | Pohl et al 2017.csv:rm | rm:r | CP-MLE | PP-LT-C | 0.053 | 0.016 | 0.007 | 0.828 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.398 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-MLE | PP-LT-C | 0.033 | 0.010 | 0.047 | 0.042 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.350 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-PB | PP-LT-C | 0.033 | 0.010 | 0.047 | 0.042 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.350 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-NPB | PP-LT-C | 0.033 | 0.010 | 0.047 | 0.042 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.350 |
| rm | Pohl et al 2017.csv:rm | rm:r | PP-B | PP-LT-C | 0.056 | 0.020 | 0.007 | 0.825 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.836 |
| quad | JinEtAl2016s2:quad | quad:ACbb1 | NP-BAYES | PP-LT-C | 0.035 | 0.018 | 0.044 | 0.156 | 0.121 | 0.034 | 0.000 | 0.000 | 0.000 | 0.009 | 1.988 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-MLE | PP-LT-C | 0.001 | 0.001 | 0.035 | 0.000 | 0.001 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 1.755 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-PB | PP-LT-C | 0.001 | 0.001 | 0.035 | 0.000 | 0.001 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 1.755 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-NPB | PP-LT-C | 0.001 | 0.001 | 0.035 | 0.000 | 0.001 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 1.755 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | CP-Bayes | PP-LT-C | 0.010 | 0.009 | 0.047 | 0.019 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 1.113 |
| rm | HilbigPohl09exp2.csv:rm | rm:b | NP-BAYES | PP-LT-C | 0.010 | 0.010 | 0.029 | 0.671 | 0.681 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 1.009 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | CP-MLE | PP-LT-C | 0.008 | 0.009 | 0.047 | 0.018 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 0.928 |
| rm | Hilbig et al 2015_exp3.csv:rm | rm:a | NP-BAYES | PP-LT-C | 0.013 | 0.014 | 0.017 | 0.690 | 0.703 | 0.062 | 0.000 | 0.062 | 0.000 | 0.031 | 0.888 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:A | CP-MLE | PP-LT-C | 0.008 | 0.009 | 0.047 | 0.254 | 0.246 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 0.853 |
| 2htsm_4 | S2002_E2:2htsm_4 | 2htsm_4:D | NP-BAYES | PP-LT-C | 0.014 | 0.017 | 0.042 | 0.904 | 0.918 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.848 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:A | CP-Bayes | PP-LT-C | 0.008 | 0.009 | 0.047 | 0.254 | 0.246 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 0.843 |
| quad | JinEtAl2016s2:quad | quad:ACbb1 | NP-PB | PP-LT-C | 0.012 | 0.018 | 0.044 | 0.133 | 0.121 | 0.034 | 0.000 | 0.000 | 0.000 | 0.009 | 0.673 |
| quad | JinEtAl2016s2:quad | quad:ACbb1 | NP-NPB | PP-LT-C | 0.012 | 0.018 | 0.044 | 0.133 | 0.121 | 0.034 | 0.000 | 0.000 | 0.000 | 0.009 | 0.673 |
| rm | HilbigPohl2008exp5.csv:rm | rm:a | NP-BAYES | PP-LT-C | 0.006 | 0.010 | 0.041 | 0.588 | 0.595 | 0.059 | 0.020 | 0.059 | 0.000 | 0.035 | 0.596 |
| quad | JinEtAl2016s2:quad | quad:ACbb1 | NP-MLE | PP-LT-C | 0.010 | 0.018 | 0.044 | 0.131 | 0.121 | 0.034 | 0.000 | 0.000 | 0.000 | 0.009 | 0.564 |
| rm | Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm | rm:a | CP-MLE | PP-LT-C | 0.005 | 0.008 | 0.043 | 0.522 | 0.527 | 0.172 | 0.086 | 0.155 | 0.000 | 0.103 | 0.542 |
| rm | Pohl, Michalkiewicz, Erdfelder, Hilbig 2017_Exp1.csv:rm | rm:a | CP-Bayes | PP-LT-C | 0.005 | 0.009 | 0.043 | 0.522 | 0.527 | 0.172 | 0.086 | 0.155 | 0.000 | 0.103 | 0.540 |
| rm | HilbigPohl09exp2.csv:rm | rm:b | NP-MLE | PP-LT-C | 0.006 | 0.012 | 0.029 | 0.675 | 0.681 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 0.513 |
| rm | HilbigPohl09exp2.csv:rm | rm:b | NP-PB | PP-LT-C | 0.006 | 0.012 | 0.029 | 0.675 | 0.681 | 0.097 | 0.042 | 0.097 | 0.014 | 0.062 | 0.513 |
| model2 | dataset | parameter | cond_x | cond_y | abs_dev | se_c | rhos_max | x | y | prop_ns_noasy | prop_ns_trait | prop_ns_nopb | prop_ns_nonpb | prop_ns | z_abs_dev |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| hb | Coolin2015.csv:hb | hb:c | NP-BAYES | PP-LT-C | 0.246 | 0.016 | 0.061 | 0.248 | 0.002 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 15.572 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-BAYES | PP-LT-C | 0.061 | 0.007 | 0.047 | 0.070 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 8.282 |
| 2htsm_5d | Mieth et al. (2016) Ex2:2htsm_5d | 2htsm_5d:d_2 | NP-BAYES | PP-LT-C | 0.356 | 0.045 | 0.058 | 0.437 | 0.080 | 0.281 | 0.010 | 0.042 | 0.000 | 0.083 | 7.888 |
| pm | Smith & Hunt 2013.csv:pm | pm:M | NP-BAYES | PP-LT-C | 0.154 | 0.020 | 0.081 | 0.831 | 0.985 | 0.000 | 0.000 | 0.200 | 0.160 | 0.090 | 7.636 |
| 2htsm_6e | Bell et al. (2015) Ex2:2htsm_6e | 2htsm_6e:d_1 | NP-BAYES | PP-LT-C | 0.351 | 0.047 | 0.084 | 0.448 | 0.097 | 0.982 | 0.009 | 0.026 | 0.000 | 0.254 | 7.530 |
| 2htsm_4 | M2003:2htsm_4 | 2htsm_4:d | NP-BAYES | PP-LT-C | 0.269 | 0.038 | 0.069 | 0.321 | 0.052 | 0.061 | 0.000 | 0.030 | 0.000 | 0.023 | 7.093 |
| 2htsm_4 | M2003:2htsm_4 | 2htsm_4:d | NP-BAYES | PP-LT-C | 0.259 | 0.041 | 0.070 | 0.327 | 0.068 | 0.061 | 0.000 | 0.030 | 0.000 | 0.023 | 6.334 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.030 | 0.005 | 0.035 | 0.031 | 0.001 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 6.217 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.092 | 0.016 | 0.007 | 0.789 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 5.929 |
| 2htsm_4 | MH2001:2htsm_4 | 2htsm_4:d | NP-BAYES | PP-LT-C | 0.208 | 0.039 | 0.079 | 0.253 | 0.045 | 0.100 | 0.000 | 0.000 | 0.000 | 0.025 | 5.338 |
| 2htsm_4 | MH2001:2htsm_4 | 2htsm_4:d | NP-BAYES | PP-LT-C | 0.228 | 0.044 | 0.077 | 0.285 | 0.057 | 0.100 | 0.000 | 0.000 | 0.000 | 0.025 | 5.237 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.161 | 0.033 | 0.025 | 0.390 | 0.229 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 4.878 |
| 2htsm_4 | BK2011_E1:2htsm_4 | 2htsm_4:b | NP-BAYES | PP-LT-C | 0.227 | 0.047 | 0.084 | 0.330 | 0.103 | 0.042 | 0.000 | 0.083 | 0.000 | 0.031 | 4.790 |
| rm | HilbigSchollPohl2010exp2.csv:rm | rm:r | PP-B | PP-LT-C | 0.015 | 0.003 | 0.035 | 0.016 | 0.001 | 0.882 | 0.765 | 0.824 | 0.000 | 0.618 | 4.708 |
| pm | Smithetal2014_Exp2.csv:pm | pm:C2 | NP-BAYES | PP-LT-C | 0.066 | 0.015 | 0.071 | 0.837 | 0.902 | 0.000 | 0.000 | 0.067 | 0.022 | 0.022 | 4.365 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | PP-B | PP-LT-C | 0.176 | 0.041 | 0.025 | 0.406 | 0.229 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 4.260 |
| hb | Coolin2015.csv:hb | hb:c | PP-B | PP-LT-C | 0.016 | 0.004 | 0.061 | 0.018 | 0.002 | 0.047 | 0.031 | 0.094 | 0.000 | 0.043 | 4.097 |
| hb | GroßBayen2015.csv:hb | hb:b | NP-BAYES | PP-LT-C | 0.113 | 0.030 | 0.048 | 0.299 | 0.412 | 0.022 | 0.022 | 0.022 | 0.000 | 0.016 | 3.696 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | PP-B | PP-LT-C | 0.033 | 0.009 | 0.047 | 0.042 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.624 |
| rm | HilbigPohl09exp3.csv:rm | rm:r | NP-BAYES | PP-LT-C | 0.076 | 0.021 | 0.086 | 0.716 | 0.792 | 0.048 | 0.016 | 0.032 | 0.000 | 0.024 | 3.559 |
| rm | Pohl et al 2017.csv:rm | rm:r | CP-Bayes | PP-LT-C | 0.054 | 0.016 | 0.007 | 0.827 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.425 |
| rm | Pohl et al 2017.csv:rm | rm:r | CP-MLE | PP-LT-C | 0.053 | 0.016 | 0.007 | 0.828 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 3.398 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-MLE | PP-LT-C | 0.033 | 0.010 | 0.047 | 0.042 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.350 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-PB | PP-LT-C | 0.033 | 0.010 | 0.047 | 0.042 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.350 |
| pd_s | stahl_2015_exp1.csv:pd_s | pd_s:C | NP-NPB | PP-LT-C | 0.033 | 0.010 | 0.047 | 0.042 | 0.009 | 0.455 | 0.030 | 0.091 | 0.030 | 0.152 | 3.350 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-MLE | PP-LT-C | 0.148 | 0.045 | 0.025 | 0.377 | 0.229 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 3.319 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-PB | PP-LT-C | 0.148 | 0.045 | 0.025 | 0.377 | 0.229 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 3.319 |
| rm | Data from Filevich Horn Kuehn 2017_within2.csv:rm | rm:r | NP-NPB | PP-LT-C | 0.148 | 0.045 | 0.025 | 0.377 | 0.229 | 0.182 | 0.101 | 0.141 | 0.000 | 0.106 | 3.319 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-PB | PP-LT-C | 0.061 | 0.021 | 0.007 | 0.820 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.873 |
| rm | Pohl et al 2017.csv:rm | rm:r | NP-NPB | PP-LT-C | 0.061 | 0.021 | 0.007 | 0.820 | 0.881 | 0.012 | 0.012 | 0.062 | 0.025 | 0.028 | 2.873 |